Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RCA] [Recent events] Create API endpoint to get events #192947

Merged
merged 18 commits into from
Sep 17, 2024
Merged

[RCA] [Recent events] Create API endpoint to get events #192947

merged 18 commits into from
Sep 17, 2024

Conversation

benakansara
Copy link
Contributor

@benakansara benakansara commented Sep 14, 2024 •
edited by kibanamachine
Loading

Closes https://github.com/elastic/observability-dev/issues/3924
Closes https://github.com/elastic/observability-dev/issues/3927

This PR introduces an events API (/api/observability/events) that will fetch -

  • All the "point in time" annotations from observability-annotations index. This includes both manual and auto (e.g. service deployment) annotations
  • The annotations will be filtered with supported source fields (host.name, service.name, slo.id, slo.instanceId) when specified as filter
  • Alerts that newly triggered on same source in given time range. The source needs to be specified as filter, when no filter is specified all alerts triggered in given time range will be returned

Testing

  • Create annotations (APM service deployment annotations and annotations using Observability UI)
  • Generate some alerts
  • API call should return annotations and alerts, example API requests
    • http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"annotation.type":"deployment"}
    • http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"slo.id":"*"}
    • http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"host.name":"host-0"}
    • http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z

@benakansara benakansara added the Team:obs-ux-management Observability Management User Experience Team label Sep 14, 2024
@benakansara benakansara self-assigned this Sep 14, 2024
@obltmachine
Copy link

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@benakansara benakansara marked this pull request as ready for review September 16, 2024 09:12
@benakansara benakansara requested review from a team as code owners September 16, 2024 09:12
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

@benakansara benakansara added the release_note:skip Skip the PR/issue when compiling release notes label Sep 16, 2024
@benakansara
Copy link
Contributor Author

run docs-build

@botelastic botelastic bot added the ci:project-deploy-observability Create an Observability project label Sep 16, 2024
shahzad31
shahzad31 reviewed Sep 16, 2024
View reviewed changes
x-pack/plugins/observability_solution/investigate_app/server/services/get_events.ts Outdated
Comment on lines 69 to 77
const response = await unwrapEsResponse(
esClient.search(
{
index: annotationsClient.index,
body,
},
{ meta: true }
)
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it would make sense to use annotationClient find method here?

Copy link
Contributor Author

@benakansara benakansara Sep 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The find method currently doesn't accept host.name and in future we might also want to support other types of sources. Also, it fetches 10000 documents, I limited it to 100 for now. So it looks like we need to either update find method or keep our custom implementation. I think current implementation in PR is more flexible as it will support all types of sources without any modifications needed in future. Wdyt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will recommend making the required changes on find method since we own it as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the find method to support filter and size

kdelemme
kdelemme reviewed Sep 16, 2024
View reviewed changes
x-pack/plugins/observability_solution/investigate_app/server/services/get_alerts_client.ts
]);

if (!alertsIndices || isEmpty(alertsIndices)) {
throw Error('No alert indices exist');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always learning about JS... throw Error() works as throw new Error()

shahzad31
shahzad31 approved these changes Sep 17, 2024
View reviewed changes
Copy link
Contributor

@shahzad31 shahzad31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !!

@maryam-saeidi
Copy link
Member

Alerts that newly triggered on same source in given time range. The source needs to be specified as filter, when no filter is specified all alerts triggered in given time range will be returned

Does it mean for the alerts without group by field, we will not call this API or we will show all alerts in the specified time range? I am trying to understand what alert annotations will be shown when there is no group by field.

@benakansara
Copy link
Contributor Author

Does it mean for the alerts without group by field, we will not call this API or we will show all alerts in the specified time range? I am trying to understand what alert annotations will be shown when there is no group by field.

If there is no group by field (and therefore we don't pass anything to filter when calling the API), it will return all alerts that newly triggered in the specified time range.

@benakansara benakansara enabled auto-merge (squash) September 17, 2024 08:58
@kibana-ci
Copy link
Collaborator

kibana-ci commented Sep 17, 2024 •
edited
Loading

💚 Build Succeeded

  • Buildkite Build
  • Commit: 6695735
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-192947-66957356967a

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
investigateApp 556 559 +3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/investigation-shared 66 73 +7

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
investigateApp 461.6KB 462.4KB +814.0B
Unknown metric groups

API count

id before after diff
@kbn/investigation-shared 66 73 +7

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @benakansara

@benakansara benakansara merged commit 808212e into elastic:main Sep 17, 2024
22 checks passed
@kibanamachine kibanamachine added v9.0.0 backport:skip This commit does not require backporting labels Sep 17, 2024
@benakansara benakansara added auto-backport Deprecated - use backport:version if exact versions are needed v8.16.0 backport:version Backport to applied version labels and removed backport:skip This commit does not require backporting labels Sep 19, 2024
@kibanamachine kibanamachine mentioned this pull request Sep 19, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 19, 2024
@benakansara
[RCA] [Recent events] Create API endpoint to get events (elastic#192947)
1407654 
Closes elastic/observability-dev#3924
Closes elastic/observability-dev#3927

This PR introduces an events API (`/api/observability/events`) that will
fetch -
- All the "point in time" annotations from` observability-annotations`
index. This includes both manual and auto (e.g. service deployment)
annotations
- The annotations will be filtered with supported source fields
(host.name, service.name, slo.id, slo.instanceId) when specified as
`filter`
- Alerts that newly triggered on same source in given time range. The
source needs to be specified as `filter`, when no filter is specified
all alerts triggered in given time range will be returned

### Testing
- Create annotations (APM service deployment annotations and annotations
using Observability UI)
- Generate some alerts
- API call should return annotations and alerts, example API requests
-
`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"annotation.type":"deployment"}`
-
`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"slo.id":"*"}`
-
`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={"host.name":"host-0"}`
-
`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z`

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 808212e)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Sep 19, 2024
@kibanamachine @benakansara
[8.x] [RCA] [Recent events] Create API endpoint to get events (#192947)…
438284e 
… (#193463)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[RCA] [Recent events] Create API endpoint to get events
(#192947)](#192947)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Bena
Kansara","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-17T09:50:09Z","message":"[RCA]
[Recent events] Create API endpoint to get events (#192947)\n\nCloses
https://github.com/elastic/observability-dev/issues/3924\r\nCloses
https://github.com/elastic/observability-dev/issues/3927\r\n\r\nThis PR
introduces an events API (`/api/observability/events`) that
will\r\nfetch -\r\n- All the \"point in time\" annotations from`
observability-annotations`\r\nindex. This includes both manual and auto
(e.g. service deployment)\r\nannotations\r\n- The annotations will be
filtered with supported source fields\r\n(host.name, service.name,
slo.id, slo.instanceId) when specified as\r\n`filter`\r\n- Alerts that
newly triggered on same source in given time range. The\r\nsource needs
to be specified as `filter`, when no filter is specified\r\nall alerts
triggered in given time range will be returned\r\n\r\n### Testing\r\n-
Create annotations (APM service deployment annotations and
annotations\r\nusing Observability UI)\r\n- Generate some alerts\r\n-
API call should return annotations and alerts, example API
requests\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"annotation.type\":\"deployment\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"slo.id\":\"*\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"host.name\":\"host-0\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z`\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"808212e97e413216655aaa9e755c671656decb46","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","auto-backport","ci:project-deploy-observability","Team:obs-ux-management","v8.16.0","backport:version"],"title":"[RCA]
[Recent events] Create API endpoint to get
events","number":192947,"url":"https://github.com/elastic/kibana/pull/192947","mergeCommit":{"message":"[RCA]
[Recent events] Create API endpoint to get events (#192947)\n\nCloses
https://github.com/elastic/observability-dev/issues/3924\r\nCloses
https://github.com/elastic/observability-dev/issues/3927\r\n\r\nThis PR
introduces an events API (`/api/observability/events`) that
will\r\nfetch -\r\n- All the \"point in time\" annotations from`
observability-annotations`\r\nindex. This includes both manual and auto
(e.g. service deployment)\r\nannotations\r\n- The annotations will be
filtered with supported source fields\r\n(host.name, service.name,
slo.id, slo.instanceId) when specified as\r\n`filter`\r\n- Alerts that
newly triggered on same source in given time range. The\r\nsource needs
to be specified as `filter`, when no filter is specified\r\nall alerts
triggered in given time range will be returned\r\n\r\n### Testing\r\n-
Create annotations (APM service deployment annotations and
annotations\r\nusing Observability UI)\r\n- Generate some alerts\r\n-
API call should return annotations and alerts, example API
requests\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"annotation.type\":\"deployment\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"slo.id\":\"*\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"host.name\":\"host-0\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z`\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"808212e97e413216655aaa9e755c671656decb46"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192947","number":192947,"mergeCommit":{"message":"[RCA]
[Recent events] Create API endpoint to get events (#192947)\n\nCloses
https://github.com/elastic/observability-dev/issues/3924\r\nCloses
https://github.com/elastic/observability-dev/issues/3927\r\n\r\nThis PR
introduces an events API (`/api/observability/events`) that
will\r\nfetch -\r\n- All the \"point in time\" annotations from`
observability-annotations`\r\nindex. This includes both manual and auto
(e.g. service deployment)\r\nannotations\r\n- The annotations will be
filtered with supported source fields\r\n(host.name, service.name,
slo.id, slo.instanceId) when specified as\r\n`filter`\r\n- Alerts that
newly triggered on same source in given time range. The\r\nsource needs
to be specified as `filter`, when no filter is specified\r\nall alerts
triggered in given time range will be returned\r\n\r\n### Testing\r\n-
Create annotations (APM service deployment annotations and
annotations\r\nusing Observability UI)\r\n- Generate some alerts\r\n-
API call should return annotations and alerts, example API
requests\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"annotation.type\":\"deployment\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"slo.id\":\"*\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z&filter={\"host.name\":\"host-0\"}`\r\n-\r\n`http://localhost:5601/kibana/api/observability/events?rangeFrom=2024-09-01T19:53:20.243Z&rangeTo=2024-09-19T19:53:20.243Z`\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"808212e97e413216655aaa9e755c671656decb46"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Bena Kansara <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kdelemme kdelemme kdelemme left review comments

@jloleysens jloleysens jloleysens approved these changes

@shahzad31 shahzad31 shahzad31 approved these changes

Assignees

@benakansara benakansara

Labels
auto-backport Deprecated - use backport:version if exact versions are needed backport:version Backport to applied version labels ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:obs-ux-management Observability Management User Experience Team v8.16.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@benakansara @obltmachine @elasticmachine @maryam-saeidi @kibana-ci @kibanamachine @kdelemme @shahzad31 @jloleysens